[SPARK-33283][CORE] Remove useless externalBlockStoreSize from RDDInfo #30179
Conversation
|
@dongjoon-hyun I'm not sure if line 1294 in spark/project/MimaExcludes.scala Line 1294 in 9d5e48e |
|
@LuciferYang, can you open a separate JIRA to manage separately? e.g.) fixed versions should be set differently. |
LuciferYang
changed the title
|
@HyukjinKwon done, reuse this pr or need re-submit one? EDIT: SPARK-33283 |
|
I believe we should keep the mima. They are removed signatures. |
|
@HyukjinKwon ok ~ keep the mima ~ thx ~ |
|
Kubernetes integration test starting |
|
Kubernetes integration test status failure |
|
Test build #130396 has finished for PR 30179 at commit
|
| info.numCachedPartitions, bytesToString(info.memSize), | ||
| bytesToString(info.externalBlockStoreSize), bytesToString(info.diskSize))) | ||
| " CachedPartitions: %d; MemorySize: %s; DiskSize: %s".format( | ||
| info.numCachedPartitions, bytesToString(info.memSize), bytesToString(info.diskSize))) |
There was a problem hiding this comment.
Since it was removed at toString, it's okay to remove it from toDebugString.
There was a problem hiding this comment.
+1, LGTM. Thank you, @LuciferYang and @HyukjinKwon .
cc @cloud-fan .
|
thx @dongjoon-hyun and @HyukjinKwon |
What changes were proposed in this pull request?
"external block store" API was removed after SPARK-12667,
externalBlockStoreSizeinRDDInfolooks like always 0 and useless. So this pr just to remove this useless variable.Why are the changes needed?
remove useless variable.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Pass the Jenkins or GitHub Action